fyi...this is a cached attribute so only hit once, but it is valid

the infer_dtype routines does do this optimization however

fyi...this is a cached attribute so only hit once, but it is valid

I know that. Unfortunately, the cache doesn't help in case of slicing: the result is always a new object (unless the slice was itself cached), the cache is not calculated yet and the value is required during construction unconditionally (fastpath=True doesn't help).

the infer_dtype routines does do this optimization however

I'm not sure what you mean. lib.infer_dtype needs to know the exact type and thus is forced to scan all the elements. is_all_dates is only interested if all of the series elements are datetimes and the first element which doesn't fit is enough to have the answer.

look at the impl of infer_dtype it fails fast. (as does is_datetime_array)

Hmmm... can it be that you confuse these two cases:

In [50]: arr_s = np.array(map(str, range(10000)))

In [51]: arr_o = np.array(map(str, range(10000)), dtype=object)

In [52]: timeit pd.lib.infer_dtype(arr_s)
1000000 loops, best of 3: 351 ns per loop

In [53]: timeit pd.lib.infer_dtype(arr_o)
10000 loops, best of 3: 42 µs per loop

In [54]: pd.lib.infer_dtype(arr_s)
Out[54]: 'string'

In [55]: pd.lib.infer_dtype(arr_o)
Out[55]: 'string'

Because, I cannot see how infer_dtype can return 'string' for arr_o without scanning the whole array at line 124 and returning the value at line 125:

    elif PyString_Check(val):
        if is_string_array(values):
            return 'string'

no that is correct. The is_string_array like all of the array functions fails fast. Of course they DO scan the entire array; but some cases are very optimized, e.g. if its already dtype = datetime64[ns] then it doesn't even need to do anything.

Pls run a perf-check with the change and see if anything changes. and post the results

test_perf.sh

Here's the result. Is there a benchmark for slice indexing?

look at vb_suite/indexing.py

so you prob need to create a benchmark which tests this change....nothing is obviously improved (but its a worthwhile change though)

you can use the example from above that you did

Done and updated the gist.

nice!

ok just need an entry in release notes (ref this pr as their is no associated issue) and good 2 go

Done.

@immerrr thanks.....

sorry about being pendantic....but expect a lot of contributions from you!